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SIGNIFICANCE  AND  EXPLANATION 


A!  * 

Consider  the  problem  of  finding  one  or  all  solutions  to  systems  of 

jji  1 

equations,  equilibrium,  fixed  points,  or  to  dynamical  systems.  In  the 

A 

last  few  years,  a  new  method  has  emerged  for  solving  this  problem.  The 
idea  is  to  start  at  a  given  solution  of  a  simpler  problem  and  to  follow 
a  path  of  solutions  as  the  path  parameter  (and  hence,  the  problem)  is 
gradually  changed.  This  path  is  proved  to  exist  via  topological  approaches 
and  is  shown  to  lead  to  the  '^right^  place. 

In  this  paper,  we  desciit)^*  a  general  predictor-corrector  method^for 
following  the  curve.  It  is  a  globalization  of  the  classical  Davidenko 
approach.  — sho*^  that  the  method  can  follow  the  path  to  any  desired 


degree  of  accuracy  and  is  convergent. 
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ON  A  PATH  FOLLOWING  METHOD  FOR  SYSTEM'.:  OF  EQUATION: 


C.  B.  Garcia  and  T.  Y.  Li 


SI.  INTRODUCTION 


In  the  last  few  years,  a  number  of  methods  |1,  3,  6-9,  12,  1 S- 17)  have  been 


p  <_  p  p  satisfying 


suggested  for  fol lowing  a  curve  y(p) 


K  in  a  given  twice  continuously  differentiable  function.  The  map 


where  H  !  H 


11  is  usually  of  a  certain  genre,  e.g.  the  fixed  point  homoto] 


or  the  Newton  homotop; 


where  x  f  R  is  some  given  starting  point.  Generally,  however,  H  need  not  be  a 


homotopy,  but  rather  H  could  be  a  system  which  describes  a  state  dynamically  changing 


overtime  [81 


These  methods  are  globalizations  of  the  Davidenko  approach  |2,  4,  5,  13,  21]  for 


following  a  path  y(t)  »  (x(t),t)  satisfying 


The  new  methods  differ  from  the  Davidenko  approach  in  that  the  path  need  not  be  para 


meterizable  in  the  last  variable  t.  In  the  new  approaches,  the  paths  are  first  proved 


to  exist  and  lead  to  the  "right"  place  whereas  in  the  Davidenko  approach  the  paths  are 


simply  assumed  to  exist  and  methods  are  described  for  following  them 


In  this  paper,  we  describe  a  general  predictor-corrector  procedure  for  following 


a  path.  Such  a  procedure  relates  to  methods  as  early  as  LahaYe  1 13]  and  Haselgrove  110] 


On  leave  from  the  University  of  Chicago.  Research  supported  in  part 
by  the  National  Science  Foundation  under  Grant  No.  MCS77-X5509  and 
the  United  States  Army  under  Contract  No.  DAAG29-7R-G-0160. 


On  leave  from  Michigan  State  University.  This  material  is  based  upon 
work  supported  by  the  National  Science  Foundation  under  Grant  Nos. 
MCS7R-0952S  and  MCS78-02420. 


Sponsored  by  the  United  States  Army  under  Contract  No.  DAAG29-7S-0-0024 


*i.' .  A  PREP  ICTUK-CVKKtXIVK  ALGORITHM 


we  say  its  ileuv.it  iv«'  ..it  is f  it's  a  Lipsohitr 


cond  v  t  ion  on  K 


if  there  exists  a  K  ■»  0  such  that  tot  all 


whore  H 


is  the  usual  Euclidean  none  and  K 


is  called  the  Llpschitr  constant 


last  P  be  an  open  Interval  containing  (p 


for  some  sealats 


be  a  smooth  curve  such  that  for  p  *  P,  we  have 


1  for  p  t  P  (parametrised  according  to  arc  length)  where 


has  rank  n  for 


all  points  on  the  curve 


are  regular 


Suppose  our  interest  is  to  follow  the  curve  y(p>,  p  <  r  >  p.  We  design  a 


predictor- cor rector  algorithm  to  follow  this  curve.  It  proceeds  as  follows.  Let 


y  “  Y (P  )•  First,  the  vector  y  is  computed  directly  from  the  basic  differential 


uation  l ^ ) 


is  H‘  with  the  ith  column  deleted.  (We  assume  here  that  (2.3)  yields  the 


’correct  orientation 


in  the  sense  that  (2.3)  evaluated  at  y  yields  y  .  Otherwise 


the  correct  differential  equation  is 


is  the  correct  differential  equation) 


Then  we  take  any  unit  vector  b  (  R 


such  that  the  cross  product  b  y 


where  0  <  S  <_  1  is  some  tolerance  value  given  apriori 


where  h  is  some  value  0  <  h  1  given  apriori,  and  Ap  >  0  a  value  to  be  defined 


in  the  next  section.  r  is  termed  the  predictor .  Ap  is  the  maximum  step  size  that 


can  be  taken  in  the  direction  b  .  h  is  a  parameter  that  controls  the  distance 


travelled  along  b  .  The  smaller  is  the  h,  the  “closer"  will  the  curve  be  followed 


by  the  predictor-corrector  method 


Next,  we  use  the  Newton  method  to  come  back  to  the  curve.  To  accomplish  this 


,  *■  „  „n+l  „n+l  . 

define  G  :  R  -»  R  by 


Setting  k  »  0  in  (2.5),  we  calculate  the  Newton 


iterates,  where  k  •  0  and 


The  step  described  by  (2.6)  is  called  the  corrector  step.  Each  z 


farther  along"  on  the 


hyperplane  b  (x  -  z  )  »  0.  This  brings  us  to  a  point  y 


Generically,  suppose  we  are  at  y  *  y (p  ),  p  ^  p  <  p.  We  take  b  such  that 


1.  We  calculate  the  predictor  z  of  (2.4).  Then,  we  compute  the 


This  brings  us  to  a  point  y 


from  (2.6)  where  z 


is  sufficiently  near 


farther  along"  y(P).  We  terminate  the  algorithm  if  y 


y(p).  See  Figure  2.1 


Figure  2.1 


there  ate  a  number  of  ways  that  we  can  choose  b  .  For  example,  we 


C 1  von 


can  choose  b  -  y  .  This  is  the  Euler  predictor.  The  resulting  algorithm  becomes 


the  Kulet  predictor-corrector  suggested  in  t!6-17).  Then  note  that  by  *  1 


that  the  tolerance  4  «  1 


As  another  example,  we  can  choose  b 


where  e  is  the  ith  unit 


vector  and  where  i  is  such  that 


The  method  is  then  the  elevator  predictor-corrector  (9).  Note  that 


so  that 


Further,  note  that  if  b 


the  Newton  correctors 


(2.6)  reduce  to 


where  z 


In  other  words,  the  elevator  predictor-corrector  method  is  the  classical  Davidenko 


approach,  except  that  there  is  an  automatic  change  of  coordinates  when  H 


becoming  singular.  See  Figure  2.2.  (Indeed,  this  example  shows  that  our  convergence 


theory  ought  to  handle  also  the  important  case  where  4  >  0  is  any  small  number.) 


Figure  2.2 


There  ere  a  number  of  problems  that  could  arise  relative  to  the  predictor-corrector 


algorithm.  For  example,  the  hyperplane  b  (c  -  z  )  -  0  may  not  intersect  the  curve 


y(P),  whence  (2.6)  will  not  converge,  ftu  is  the  common  problem  encountered  by  a 


local  Newton  method  or  Davidenko  method.  Another  conceivable  problem  is  that  the 


sequence  {y  )  generated  might  converge  to  a  point  y*  e  y(P)  far  from  y(p),  so 


that  the  method  never  reaches  the  end  of  the  path.  Finally,  .a  serious  problem  that 


Then  we  could 


can  occur  is  that 


'wrong 


cycle  indefinitely  in  a  portion  of  the  path.  See  Figure  2.3 


In  the  next  section,  we  show  that  none  of  these  problems  can  arise.  There,  we 


show  that  the  predictor-corrector  can  be  guaranteed  to  terminate  with  a  point 


t  ,  where  *  >  0 


with 


best  of  our  knowledge,  these  problems  do  not  appear  to  have  been  thoroughly  investi 


gated  in  the  literature.  (See  also  the  remark  after  Theorem  3.1) 


Figure  2.3 


S3.  CONVERGENCE  OF  THE  PREDICTOR- CORRECTOR  METHOD 

In  this  section,  we  show  the  convergence  of  the  predictor-corrector  method. 

2 

Throughout,  we  assume  that  H  is  C  and  satisfies  a  Lipschitz  condition  (2.1).  A 
curve  y(P)  satisfying  (2.2)  is  given. 

The  first  theorem  we  show  is 
Theorem  3.1. 

kkOk“*  k  k.k 

Consider  y  ay(p),  p  ^  p  ^  p.  Then  for  any  b  such  that  by  ?  0, 

k  k  k 

b  jj  »  i,  there  exists  an  open  ball  N(y  ,  6  )  with  center  y  and  radius  5^  such 


that  for  any 


k  _  k,0 

z  -  z  f  N(y  , 5  ) 


the  Newton  correctors  (2.6)  converge  to  a  point  y  «  y(P).  Moreover,  there  exists  a 
>  0  such  that  yk+*  is  the  unique  solution  of  G  *  0  in  the  open  ball  N(z 


Proof:  Consider  G’ (z)  « 


[  H* (yk) 


.  ,  k.t  .k 

h'  (y  )  ,y 


H’ (yk)H' (yk)t  o 

.  k  ,  k  t  k.k 
b  H' (y  )  by 


and  since  [H'  (yk)t,ykl  and  the  right  hand  side  matrix  of  (3.2)  are  nonsingular. 

.  k  k  * 

G' (y  )  is  nonsingular.  Hence,  there  is  an  open  ball  N(y  ,6^)  y  with  radius 


6^  and  a  y  >  0  such  that 


j|G’(z)  1 II  <_  Y,  for  all  ze  N(yk,63) 

Next,  by  the  continuity  of  G,  for  any  e  >  0,  there  exists  an  open  ball 


N(y  , 6j)  such  that 


|G(z)  )|  <  e  for  all  z  e  N(y  ,6^) 


Lastly,  note  that  if  K  is  the  Lipschitz  constant  for  H', 


|G>  <z)  -  G*  (y) | 


,H-(z,  -H-(y)|,lK| 


the  hypotheses  of 


■  -  ....  -it  K 


Using  ( J.  1)- ( 3.  5),  we  can  now  show  that  for  appropriate  z  , 

the  Newton-Kantorovich  theorem  111,  18)  holds.  To  see  this,  given  of  (3.3) 

K  of  0.5),  choose  0  <  4.  <  &  such  that  (3.4)  holds  for  >.  ■  — is—.  Then,  tor 

1  ~  3  2vzK 


and 


Henct,  the  hypothesis  of  the  Newton-Kantorovich  theorem  holds.  Thus,  the  Newt  ,  r: 

K-f  1  K. 

iterates  12.6)  coverage  to  a  solution  y  of  G  •  0  which  is  unique  in  N(z  , 

whara  «2  •  (>K) -1 11  ♦  (1  -  ra)1/2). 

This  theorem  closely  relates  to  theorems  proved  in  116,  17].  However,  note  that 

k+1 

the  theorem  above  gives  no  information  as  to  where  on  the  curve  y  lies.  For  example, 

k+l  k 

in  Figure  2. 3,  y  could  be  a  point  as  near  as  we  please  to  z  ,  yet  is  such  that 

p*1  *  <  pn.  To  prevent  such  cases  from  occurring,  we  need  a  stronger  result. 

Consider  yk  ■  y(pk),  p°  £  pk  <  p.  Let  bk  be  such  that  ||b'l||  »  1,  bkyk  ^  0. 


Let 

g(p)  -  bky(pt  . 

By  the  Implicit  Function  theorem,  g(p)  »  b*y(p)  is  continuous  in  a  neighborhood  of 

k  k  k 

y  .  There  is  an  open  ball  N(y  , )  such  that  y(P)  n  N(y  ,M^ )  is  a  connected  smooth 

k  k 

path  containing  y  and  §(p)  0  for  any  y(p)  <  N(y  ,M1>. 

Proposition  3,2. 

For  any  0  <  Mj  <_  M^,  there  is  an  MJ(  0  <  such  that  for  any 

z*  «  tl(yk,M3) 

(i)  T(*k)  ■  (*|bk(*  -  zk)  ■  0}  intersects  y(P)  n  at  a  unique  point, 

and 

k 

(ii)  the  point  of  intersection  is  in  N(y  ,M,). 

k  k 

Proof :  Let  H(z)  »  b  z  and  let  a^.a.,  be  such  that  Q  •  y(P)  o  N(y  , M^)  • 

^ V <P> 1 ® ^  <  P  <  a,).  Then,  since  g  is  strictly  monotone  in  p  for  y(p)  f  N(yk,M 
k  k  k 

glSj)  <  glp  )  ■  h(y  )  <  g(a2>,  or  gla^  >  h(y  )  >  g(a2>.  There  is  an  open  ball 
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0  <  M  <  M  such  that  for  any 


at  a  unique  point 


(ii)  The  point  of  intersection  is  in  N(y 


be  such  that 


Then,  since  g  is  strictly  monotone  in  p  for 


an  open  ball  N(y 


M  )  such  that  g(a  )  <  h(z)  <  g(a  )  for  all  z  «  N(y 


M  )  such  that  T(z  )  n  Q 


Then  either 


That  is 


This  is  a  contradiction,  since  g  is  continuous  and  g(a  )  <h(z  )  <g(a  ) 


Finally,  T(z  )  n  y(p)  n  N(y 


M  )  is  a  unique  point  since  g  is  strictly 


monotone  in 


let  b  be  chosen  such  that  ||b 


where  6,  is  defined 


Then,  if  0  <  M_  < 


in  (3.1),  the  Newton  correctors  (2.6)  with  starting  point 


Moreover,  we  have 


converges  to  a  unique  point  y 


Jc 

Proof :  By  proposition  3.2,  there  exists  a  unique  y(p*)  r  N(y  .Mj)  such  that 

K  k 

b  (y(p+)  -  z  )  »  0.  We  first  note  that 

||y(p*)  -  zk||  <  2M2  .  (3.7) 

Note  that  y(p*)  is  a  solution  of  G  =  0  defined  by  (2.5).  Also,  by  Theorem  3.1, 

k+1  k+1 

the  Newton  iterates  converge  to  a  point  y  =  y(p  )  e  y(P). 

Suppose  yk+1  /  y(p*).  Then  yk+1  f  N(yk,M1>.  Hence  ||yk+1  -  zk||  >  M1  -  M3  >  3M2 

so  that  radius  S2  of  the  ball  N(z  ,6^)  in  Theorem  3.1  is  greater  than  From 

Jc 

(3.7),  this  implies  that  y(p*)  «  N(z  ,6  ),  a  contradiction  of  the  uniqueness  of 

)c 

solutions  for  G  =  0  in  N(z  ,  62>. 

,  ,  .k  k+1  k.  ,k,  k+1  k.  .k,  k  k.  .k ,  k  k^ 

Finally,  we  have  b(y  -y)=b(y  -  z  )  +  b  (z  -  y  )  =  b  (z  -  y  )  = 

k  k  k  k+1  k 

hM^b  b  >  0.  That  is,  for  g(p)  a  b  y(p),  we  have  g  (p  )  >  g  (p  ) .  Since  g(p)  >  0 

*  ,  \  ,  k  k+1  k 

for  y  (p)  in  N(y  ,M1>,  p  >  p  . 

Let  us  reconsider  the  predictor-corrector  method  described  in  §2.  Suppose  we  are 

k  k  Ok-  kkk 

at  a  point  y  =  y(p  ),  p  £  p  <p  on  the  path.  We  choose  b  ,  by  ^  6  >  0, 

(I bk ||  =  1  where  6  is  some  a  priori  given  tolerance.  (We  must  bound  bkyk  below 

k  k  k.  k 

by  6  so  as  to  avoid  getting  a  sequence  {b  ,y  }  where  by  ■+  0,  which  causes 

difficulties).  By  proposition  3.3,  there  is  an  My  0  <  M^  <  6^  such  that  if  the 

predictor  z  is  computed  by  (3.6),  the  Newton  correctors  (2.6)  will  come  back  to  a 

k  k 

point  farther  along  the  curve.  Let  ftp^  s  Ap(y  ,b  )  of  (2.4)  denote  the  maximum 
number  of  such  kind.  By  repeated  use  of  the  predictor-corrector  method,  we  will 
generate  a  sequence  {y  }  on  the  curve  y(P).  We  terminate  when  we  generate  a  point 
yn  ”  y(pn)  such  that  pn  ^  p  -  e  for  some  prescribed  tolerance  e  >  0. 

Theorem  3.4. 

The  predictor-corrector  method  terminates  in  a  finite  number  of  iterations. 

k  k 

In  order  to  prove  Theorem  3.4,  we  need  the  next  lemma.  Given  y  ■  y(p  ), 
p^  <_  pk  <  p  and  tolerance  6  >  0,  define 

q  «  min{Ap(yk,b)  |byk  >_  6/2,  ||b||  =  1} 


(3.8) 


where  A|>(y  is  the  maximum  n umbel  such  that  tot 

*k  a  *k,°  .  yk  ♦  hAp(yk,b)b  ( t.u) 

the  Newton  correctors  12. ti)  (where  bk  in  rept  aced  by  b)  will  come  back  to  a  |m(ii( 
fait  lux  atom)  the  |<ath>  Since  the  constraint  net  ot  (1,8)  in  compact ,  amt  since 
Apty  .hi  '  0  tot  all  b  uatistyinq  the  const  i  and  s  i't  it.lt),  we  have 

q  x  0  ,  t  l.  10) 

lemma  i.  V 

k  k  k 

Ttieie  exintn  all  open  halt  Nty  ,M)  ot  V  auch  ttiat  tor  any  y  e  Niy  ,Mt  ’  yd'l, 
and  any  b  nuoh  that  bj  '  A,  ||h||  «  1,  w«  have  Aply.b'  '  '  q. 

Pi  oof  t  Choose  0  '  M  v  |  q,  amt  M  nnwill  enouqh  auch  t hat  for  any  y  <  Ntyk,M)  11  vit't, 
we  have 

b}  >  (|  ||b||  *  l  implies  b'f*  '  S/2  .  tt.tt) 

k  k 

Hence,  by  (t.H),  Apty  ,b)  ^  q,  Moreover,  for  any  y  f  N(y  ,M>  >'  y(P),  wo  have 

Nty,  *,  q)  C  Ntyk,q).  Therefore,  by  (3.11),  for  any  b  auch  ttiat  bj  '  S,  ||b||  -  t  amt 
k  1  v  k 

any  a  «  Nty,  -q)  wo  have  a  »  Nty  ,q)  so  that  Theorem  t,  l  implies  ttiat  the 

k  k 

corrector*  (2.t>)  (where  b  is  replaced  by  b)  converge  when  started  at  »  .  Hence, 

Apty.b)  >  |  q. 

Finally,  we  may  now  prove  Theorem  3,4. 


Proof  of  Them em  J.  4. 

Suppose  the  predictor  corrector  novel  ends.  Then  ttieie  Im  an  infinite  sequence 

k  k  k 

{y  )  auch  that  y  t  y (P)  for  all  k  and  Uw  Y  -  y*  •  y ly'M  fo«  p*  v,  n  -  t . 

k  K%> 

But  since  <yk*1  -  sN  (*k  -  yN  “0,  we  tiave 


-  vKll‘  -  II V 


k+l  -  »kl|3*ll*K  -  Vkll*' 


l«k  -  yk||3"  (hAph>3 


Hence,  Vim  tyK+'  -  yk)  -  0  implies  l im  Ap^  -  0.  This  ts  a  cont radlction,  since  by 

WM»  W  MW 

R  k  R  1 

lemma  !.■>,  for  all  y  sufficiently  near  y*.  we  have  Ap^  '  ,  q. 
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